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Biologically  Motivated  Representational  Schemes  for  Mapping 
Polarization  Information  into  Visual.  In  formation 

Abstract 

Polarization  is  an  important  feature  of  electromagnetic  waves,  and  it  can  be  affected 
by  surface  shapes,  materials,  local  curvature  and  features,  and  relative  location  of  sources 
and  objects,  and  thus  it  can  provide  useful  information  about  the  observed  scene  and 
objects  Without  appropriate  instruments  human  eyes  cannot  effectively  sense  the 
polarization  of  light.  However,  it  is  well  known  that  eyes  of  certain  animal  species  are 
sensitive  to  light's  polarization.  In  addition  to  the  well-known  navigational  advantage  of 
polarization  sensitivity,  it  has  been  hypothesized  that  some  species  may  have  evolved 
polarization  sensitivity  as  a  mechanism  of  enhancing  the  contrast  of  targets  in  media  that 
scatter  light  (e.g.,  under  water). 

Motivated  and  inspired  by  hypotheses  about  utility  and  nature  of  the  polarization 
sensitivity  of  visual  systems  of  certain  species,  we  have  been  studying  and  developing 
imaging  methodologies  based  on  the  "polarization-difference"  signals.  In  our  previous 
work,  we  showed  that  polarization-differencing  greatly  facilitated  the  detection  of  targets  in 
scattering  media,  even  when  the  targets  produced  only  very  weak  polarization.  We  also 
developed  some  bio-inspired  representational  schemes  for  mapping  polarization 
information  into  visual  information,  suitable  for  human  observers  and  consistent  with 
human  visual  system  and  perception. 

In  this  project,  we  explore  the  possibility  of  utilizing  coherent  motion  of  tiny  dots 
superimposed  on  an  image  as  means  of  representing  and  displaying  polarization 
information  on  the  image,  while  maintaining  other  features  of  conventional  images.  Our 
results  show  that  image  segments  possessing  coherently  moving  dots  “pop  out”  in  contrast 
with  the  rest  of  the  image  with  randomly  moving  dots,  and  have  demonstrated  that  coherent 
motion  can  be  exploited  in  providing  visual  cues  for  polarization  contents.  The  direction  of 
motion  of  the  dots  in  each  segment  of  the  image  can  be  related  to  the  preferred  axis  of 
polarization  in  that  segment,  and  the  degree  of  coherence  can  be  used  for  representing  the 
degree  of  linear  polarization.  In  this  report,  first  a  brief  background  on  the  polarization 
vision  will  be  given,  and  then  our  efforts  on  mapping  polarization  information  onto  visual 
cues  using  coherently  moving  dots  will  be  presented. 
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Biologically  Motivated^  Representational^  Schemes  for  Mopping 
Polarization  Information  into  Visual.  Information 

Background  and  Motivation 

It  is  well  known  that  wave-based  navigation,  direction  finding,  and  imaging  and 
detection  of  objects  in  scattering  media  present  a  challenging  set  of  problems.  The 
scattering  of  electromagnetic  waves  in  media  surrounding  an  object  is  a  serious 
impediment  to  the  proper  function  of  imaging,  detection,  and  navigational  systems  [1-4]. 
In  the  visual  and  infrared  domains  scattering  due  to  suspended  particulates  in  water  and 
air  in  the  form  of  fog,  rain  and  smoke  can  seriously  reduce  target  visibility  [5],  due  to  the 
reduction  of  target  contrast  at  the  imaging  system  by  light  scattered  from  the  particles. 
Introducing  novel  ideas  to  reduce  such  fundamental  limitations  will  open  doors  to  new 
methods  for  imaging,  information  gathering,  and  detection  and  navigation  when 
scattering  particles  impair  system  performance  as  in  underwater  environments  and  in  fog 
and  rain.  Most  imaging  systems  simply  map  total  intensity  spatially,  sometimes  with  a 
few  spectral  channels,  and  are  not  efficient  at  seeing  through  scattering  media,  since 
scattered  light  degrades  image  contrast.  Novel  insight  into  methods  for  seeing  targets  in 
scattering  media,  and  improving  the  quality  of  their  images  gathered  in  such 
environments,  has  come  from  examining  the  biology  of  visual  systems  of  animal  species 
thought  to  have  evolved  an  ability  to  see  in  the  presence  of  scatterers. 

The  underwater  environment  is  a  good  example  of  a  domain  in  which  scatterers 
are  present  and  limit  the  performance  of  imaging  systems.  Therefore,  it  has  been  a 
suitable  place  to  look  for  a  naturally  occurring  visual  system  that  may  have  adaptations 
for  improving  the  quality  of  vision  in  scattering  media,  and  to  attempt  to  introduce 
interdisciplinary  ideas  to  overcome  the  limitations  and  constraints  in  imaging  in 
scattering  media. 


Biological  visual  systems  have  developed  coding  strategies  that  enhance  contrast 
in  scattering  media  [6-8].  One  well-documented  strategy  is  the  widespread  use  of  long- 
wavetength  light,  as  its  lower  scattering  preserves  image  contrast  over  greater  distances 
[8-9J.  Biological  visual  systems  in  many  species  have  also  developed  schemes  to  sense 
and  utilize  the  light's  polarization  [10-17].  Polarization  vision  is  an  active  field  that 
investigates  the  biophysical  mechanisms  and  functional  roles  of  polarization  sensing  by 
biological  visual  systems  [10-16],  Polarization  itself  is  a  fundamental  characteristic  of 
electromagnetic  radiation,  and  plays  an  important  role  in  the  interaction  of  light  with 


Human  vision  is  "polar  blind,"  and  cannot  effectively  utilize  the  polarization  of 
light  even  though  polarization  carries  important  information  about  the  surface  properties 
ol  objects  and  other  aspects  of  the  environment.  Indeed,  in  our  previous  work  we  have 
shown  that  "biologically  inspired  polarization-differencing"  can  extend  the  distance  over 
which  objects  can  be  reliably  detected  in  scattering  media  up  to  3-fold,  even  under 
conditions  when  the  fractional  polarization  reaching  the  detector  is  less  than  1%  [18-20]. 
Extending  the  distance  for  detection  and  classification  of  objects  near  the  limit  of 
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visibility  increases  the  time  for  critical,  survival-relevant  decisions,  and  is  thus  a  major 
advantage  in  many  real-time  applications. 

Polarization  information  has  been  used  by  researchers  in  several  remote  sensing 
applications  ranging  from  machine  vision  [29]  to  biomedical  imaging  [30-31]  to  infrared 
target  detection.  In  addition  to  its  ability  to  provide  previously  unavailable  information 
about  scenes,  polarization-difference  imaging  (PDI)  improves  target  detectability  in 
scattering  media  [18-20].  In  our  previous  effort,  we  investigated  the  relationship  between 
polarization-sum 1  (PS)  and  polarization-difference  (PD)1  information  to  determine  if  the 
two  types  of  information  could  be  combined  into  a  single  image  to  further  enhance  target 
identification  [20].  Using  the  principal  components  analysis  of  polarimetric  information, 
fundamental  orthogonalities  were  discovered  between  the  PS  and  PD  processing  channels 
[32-33].  Use  of  this  information  coupled  with  knowledge  of  the  biology  of  human  color 
vision  [34]  led  to  the  development  of  novel  display  strategies  for  2-channel  colorimetric 
PD  images  [20],  Further  development  of  the  theory  led  to  a  fundamental  understanding 
of  the  3-channel-polarization-to-3-channel-color  mapping  first  proposed  by  Bernard  and 
Wehner  [35]. 

Figure  1  shows  one  of  the  earlier  images  in  our  previous  effort  of  an  aluminum 
disk  immersed  in  a  tank  of  water  to  which  small  amount  of  milk  is  added,  obtained  using 
a  conventional  (i.e.,  PS)  imaging  (left  panel),  the  PD  imaging  (middle  panel)  and  the. 
colorimetric  PDI  technique  (right  panel)  [20],  In  this  example,  the  PS  image  provides  us 
with  the  information  about  the  boundary  of  the  disk  but  not  about  the  patches  over  this 
disk.  The  PD  image,  on  the  other  hand,  clearly  shows  the  patches,  but  not  the  target 
boundary.  The  colorimetric  representation  of  the  PD  image  combines  all  the  information 
into  a  single  image  using  our  novel  2-channel  polarization  mapping  strategy.  By 
combining  PS  and  PD  information,  both  the  disk  (visible  in  the  PS  image)  and  the 
patches  (visible  in  the  PD  image)  are  readily  observed  in  a  single  scene.  The 
enhancement  by  the  colorimetric  PDI  mapping  of  the  visibility  of  the  weakly  (and 
orthogonally)  polarizing  square  patches  on  the  disk  in  the  image  is  striking  to  the  normal 
human  observer. 


‘A  “polarization-sum”  (PS)  image  is  created  from  summing,  pixel  by  pixel,  the  light  gathered 
(successively)  through  two  orthogonal  linear  polarizers.  So  pxl[x,y)  =  /  (x,y)  +  Ix  (x.y) .  The  pixel-by¬ 
pixel  image  is  directly  proportional  to  that  which  would  be  obtained  by  a  conventional,  polarization- 
insensitive  imaging  system.  A  “polarization-difference”  (PD)  image  is,  however,  formed  by  subtracting, 
pixel  by  pixel,  the  light  gathered  through  the  two  orthogonal  polarizers,  so  one  gets 

pni{^y)=1n(x>y)-ii{^y)- 
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Fig.  1 .  A  sample  of  our  previous  results  obtained  in  our  earlier  effort.  This  is  an  example  of  the  advantage  of 
biologically  inspired  polarization-difference  imaging.  Left  panel  shows  a  conventional  (i.e.,  polarization-sum 
PS  image)  of  a  prepared  aluminum  disk  immersed  in  a  tank  of  water  with  small  amount  of  milk  added. 
Middle  panel  shows  the  polarization-difference  (PD)  image  of  the  same  target  under  the  same  experimental 
condition.  Right  panel  shows  a  colorimetric  PDI  image  combining  the  information  obtained  from  the  PS  and 
PD  images.  The  square  patches  on  the  disk,  which  produce  less  than  2%  net  linear  polarization  in  the  image, 
are  clearly  detectable  and  discriminable  in  the  colorimetric  and  PD  images,  but  barely  detectable  and  not 
discriminable  in  the  conventional  image. 

How  Can  We  Map  Polarization  Information  into  Visual  Information/Cue? 

It  is  known  that  polarization  imaging  contains  useful  information  about  surface 
shape,  local  curvature  and  features,  and  the  relative  location  of  light  sources  in  the 
environment  with  respect  to  the  object.  However,  in  mapping  polarization  data  into 
visual  displays,  it  is  important  to  note  that  since  human  eyes  are  polar  blind,  and  yet  that 
the  outputs  of  any  imaging  and  detection  machines  will  eventually  be  viewed  by  human 
observers,  the  outputs  of  any  polarization  imagery,  detection  and  navigation  system 
should  be  exhibited  in  a  way  that  will  be  easily  perceived  by  the  human  visual  system. 
Thus,  the  information  from  polarization-based  imaging  systems  needs  to  be  rendered  or 
mapped  to  the  display  in  a  manner  compatible  with  the  biology  of  human  vision. 

Some  of  the  known  visual  cues  include: 

(1)  Intensity/Brightness 

(2)  Color 

(3)  Coherent  motion  (Shape/form  from  Motion) 

(4)  Textures 

(5)  Three-Dimensional  visualization  (shape  from  shading) 

(6)  Second-order  motion 

(7)  Other  cues 

Previously  we  had  shown  that  polarization-difference  imaging  (PDI)  information  can  be 
usefully  color-coded  (the  item  (2)  of  the  visual  cues  mentioned  above)  based  on  the 
opponent-colors  model  of  the  human  visual  system  [20],  In  such  mapping,  the 
polarization-sum  (PS)  signal  and  the  magnitude  of  polarization-difference  (PD)  signal 
were  represented  by  the  colorimetric  parameters  brightness  (B)  and  saturation  (S), 
respectively,  while  the  sign  of  PD  signal  corresponded  to  a  pair  of  opponents  hues  (H  and 
H+180°).  The  choice  for  the  specific  value  of  hue  (H)  presents  a  degree  of  freedom, 
which  could  be  determined  according  to  any  suitable  parameter.  In  our  previous  work. 
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we  had  selected  the  hue  (H)  based  on  the  value  of  the  polarization  analyzer  angle.  This 
technique,  which  appeared  to  be  simpler  than  some  other  polarization  representational 
systems  (e.g.,  [35])  and  to  be  more  suitable  for  observations  made  in  scattering  media,  led 
to  promising  and  important  design  features  for  contrast  enhancement  in  imaging  of 
objects  in  scattering  media.  Exciting  results  for  such  representational  schemes  were 
obtained  that  are  easily  understandable  and  naturally  interpretable  by  human  observers. 
(See  e.g.,  Fig.  1). 

Mapping  polarization  information  onto  intensity/brightness  can  be  done  by  displaying 
polarization-sum  (PS)  signal  and  the  magnitude  of  polarization-difference  (PD)  signal 
into  separate  black&white  images.  This  was  done,  for  example,  in  the  left  and  middle 
panels  of  Fig.  1. 

The  work  proposed  under  this  grant  was  focused  on  mapping  polarization  information 
onto  coherent  motion,  i.e.,  item  (3). 


Mapping  Polarization  Information  into  Coherently  Moving  Dots 

Human  visual  system  can  detect  and  perceive  coherent  motion,  moving  objects,  and  form 
from  motion  easily  and  pre-attentively.  Biological  motion  is  easily  perceived  when, 
signal  dots  are  placed  in  a  field  of  randomly  moving  noise  dots  [36].  Whereas  perception 
of  biological  motion  suffers  under  dim-light  conditions,  detection  of  coherent  motion 
seems  unperturbed  so  long  as  dot  size  and  density  are  sufficient  to  support  spatial 
resolution  of  the  motion  tokens  [37]. 

The  idea  of  mapping  polarization  information  onto  coherently  moving  dots  can  be 
described  as  follows:  In  a  scene  where  there  are  many  dots  randomly  located,  it  is  known 
that  coherently  moving  dots  can  be  easily  and  pre-attentively  distinguished  from 
randomly  moving  dots.  Figure  2  shows  a  collection  of  white  dots  on  a  black  background. 
These  dots  are  randomly  located.  A  section  of  this  collection  contains  dots  that  will 
move  coherently  in  a  particular  direction,  while  the  other  dots  either  move  randomly  or 
they  do  not  move  at  all.  Human  eyes  can  easily  distinguish  the  section  of  coherently 
moving  dots  against  the  background  of  randomly  moving  dots  (or  non-moving  dots). 
(Please  double  click  on  Figs  2  and  3  to  see  the  movie  clips  of  moving  dots.) 

Since  the  regions  of  coherently  moving  dots  can  be  detected  by  the  human  eye,  we 
explore  the  possibility  of  using  such  a  visual  cue  as  a  means  to  display  polarization 
information  in  an  image.  We  can  add/overlay  small  dots  on  an  image.  The  size  and 
number  of  these  dots  are  chosen  such  that  only  a  little  or  no  change  to  the  content  of  the 
original  image  is  done.  The  regions  of  image,  which  contain  some  polarization,  can  be 
the  ones  with  coherently  moving  dots,  whereas  the  regions  with  no  polarization  can 
include  dots  that  will  be  moving  randomly.  The  direction  of  movement  can  relate  to  the 
direction  of  polarization  of  the  corresponding  region.  So  regions  with  dots  moving  in 
different  directions  represent  regions  with  different  polarization.  (See  Fig.  4)  The 


regions  of  interest  may  be  the  ones  with  randomly  moving  dots,  while  the  other  section  of 
the  image  may  have  dots  that  are  not  moving.  (See  Fig.  5). 


Error!  Not  a  valid  link. 

Fig.  2.  Collection  of  randomly  located  dots.  A  section  of  this  collection  includes  dots  that  will  move 
coherently,  while  the  other  dots  will  be  moving  randomly.  Double  click  on  the  Figure  to  see  the  movie  clip. 


Error!  Not  a  valid  link. 

Fig.  3.  Similar  to  Fig.2.  except  the  dots  outside  the  section  of  coherently  moving  dots  will  stay  motionless. 
Double  click  on  the  Figure  to  see  the  movie  clip. 


Error!  Not  a  valid  link. 

Fig.  4.  Similar  to  Fig.2,  except  the  dots  in  the  two  regions  of  coherently  moving  dots  are  moving  in 
different  directions.  Double  click  on  the  Figure  to  see  the  movie  clip. 


Error!  Not  a  valid  link. 

Fig.  5.  Similar  to  Fig.2,  except  the  two  regions  of  interest  have  dots  that  are  moving  randomly.  The  rest  of  the 
image  contains  dots  that  are  not  moving.  Double  click  on  the  Figure  to  see  the  movie  clip. 

Brightness  intensity  of  the  dots  is  chosen  such  that  the  dots  can  be  easily  observed 
against  the  image.  Here  we  can  use  two  different  ways  of  choosing  brightness  intensity  of 
the  dots.  The  first  one  is  the  “maximum  contrast  scheme”,  where  brightness  intensity  is 
selected  complement  to  the  intensity  of  the  corresponding  nearby  pixels  of  the  original 
image.  The  second  scheme  'is  the  “intensity  modulation  scheme”.  In  this  case  the 
brightness  intensity  of  the  dot  is  chosen  according  to  the  local  intensity  of  the  image 
using  the  following  relation, 

L,  =  L,A'-M/m) 

where  lJm  and  are  the  brightness  intensity  of  dot  and  the  local  image  pixels, 
respectively,  and  M  is  the  given  percentage  rate  of  intensity  modulation. 

To  explore  and  test  this  idea,  an  original  algorithm  has  been  developed  in  the 
MATLAB  environment.  The  results  are  presented  as  movie  in  *.mpg  format.  (These 
movie  files  can  be  viewed  by  Microsoft  Media  Player. 
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As  an  illustration  of  this  technique,  we  use  our  target,  which  is  an  aluminum  disk  with 
two  square  patches  on  its  surface.  This  is  a  3.8-cm-diameter  aluminum  disk  whose 
surface  has  been  sandblasted  to  make  it  Lambertian,  except  for  two  1-cnT  square  patches, 
which  are  not  sandblasted,  but  instead  they  are  "sand-papered”  in  two  orthogonal 
directions,  resulting  in  small  polarization  signatures. 


Fig.  6.  One  of  the  targets  under  test 

Now  we  present  images  of  this  target  with  moving  dots  inserted  to  display  the 
polarization  information.  Figures  7  and  8  present  the  mapping  of  polarization 
information  of  this  image  onto  the  coherently  moving  dots  with  two  different  directions 
of  motion.  In  Fig.  7,  the  region  with  no  polarization  information  is  represented  by 
randomly  moving  dots,  whereas  in  Fig.  8  such  a  region  contains  dots  that  are  not  moving. 
In  these  two  Figures,  the  brightness  intensity  of  dots  is  chosen  according  to  the 
“maximum  contrast  scheme”.  Figures  9  and  10  show  similar  images,  but  for  the 
“intensity  modulation  scheme”  with  30%  modulation.  Figures  11  and  12  present  the 
images  of  this  target,  but  here  the  dots  in  the  polarization  regions  are  being  “stretched”, 
instead  of  being  moved.  The  direction  of  the  “stretching”  can  map  and  relate  to  the 
direction  of  polarization.  Such  “stretching”  of  dots,  and  the  regions  in  which  these  dots 
are  located,  can  be  easily  detected  by  human  eyes. 

Error!  Not  a  valid  link. 

Fig.  7.  Image  of  the  target  disk,  with  coherently  moving  dots  indicating  regions  with 
polarization  information.  The  dots  in  these  two  regions  are  moving  in  orthogonal  directions, 
mapping  two  orthogonal  directions  of  polarization.  Double  click  the  Figure  to  see  the  movie. 

Here  “maximum  contrast  scheme”  is  used  for  brightness  intensity  of  dots. 


Error!  Not  a  valid  link. 

Fig.  8.  Similar  to  Fig.  7,  except  the  dots  in  the  region  with  no  polarization  information  do  not  move. 

Double  click  on  the  Figure  to  see  the  movie 


Error!  Not  a  valid  link. 


Fig.  9.  Similar  to  Fig.  7,  except  here  the  “intensity  modulation  scheme”  is  used  to  assign  the  brightness 
Intensity  for  dots.  Double  click  on  the  Figure  to  see  the  movie 


Error!  Not  a  valid  link. 

Fig.  10.  Similar  to  Fig.  9,  except  the  dots  in  the  region  with  no  polarization  information  do  not  move. 

Double  click  on  the  Figure  to  see  the  movie 


Error!  Not  a  valid  link. 

Fig.  11.  Similar  to  Fig.  7,  except  here  the  dots  in  the  polarized  regions  are  being  “stretched”,  instead  of 
being  moved.  Double  click  on  the  Figure  to  see  the  movie. 


Error!  Not  a  valid  link. 

Fig.  12.  Similar  to  Fig.  1 1,  but  here  the  “intensity  modulation  scheme”  is  used  for  the  brightness  intensity 
of  dots.  Double  click  on  the  Figure  to  see  the  movie. 


It  is  important  to  note  that  the  “maximum  contrast  scheme”  for  the  dot  intensity, 
works  better  when  image  intensity  is  more  uniform,  whereas  the  “intensity  modulation 
scheme”  is  more  preferable  for  the  situations  where  certain  parts  of  the  image  represent 
unimportant  background  whose  intensity  is  close  to  zero.  In  these  cases,  this  scheme 
enables  one  to  concentrate  attention  on  the  region  of  interest,  instead  of  background. 


Using  Textures  for  Mapping  Polarization  Information  into  Visual 
Information 

We  have  also  explored  the  possibility  of  using  black-and-white  (BW)  textures  to  display 
and  map  polarization  data.  Figures  13  shows  the  use  of  textures  (both  static  and 
dynamic  textures)  in  displaying  regions  with  orthogonal  polarization  information.  The 
textures  are  formed  by  25%,  10%  and  5%  intensity  modulation  in  the  region  with 
polarization  information.  As  can  be  seen  from  these  Figures,  the  textured  areas,  which 
represent  the  areas  with  orthogonal  polarization  axes,  can  be  easily  detected  by  the 
human  eyes. 

Error!  Not  a  valid  link.  (A)  Error!  Not  a  valid  link.(B) 


Error!  Not  a  valid  link.(C) 

Fig.  13.  Use  of  textures  to  show  polarization  information  in  our  target  disk.  The  percentage  of 
modulation  for  these  textures  are  25%  in  A,  10%  in  B,  and  5%  in  C.  Double  click  on  the  Figures  to 
see  the  movies  for  “dynamic”  texture  mapping  of  polarization. 
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We  have  also  begun  making  a  Graphical-User-Interface  (GUI)  panel  to  include 
some  of  the  mapping  strategies  and  representational  schemes  we  have  developed  in  this 
program  (and  the  other  mapping  schemes  we  continue  to  develop  in  our  current  grant),  in 
order  to  make  the  selection  of  a  specific  mapping  technique  easier.  Figure  14  shows  a 
sketch  of  this  GUI.  As  can  be  seen,  in  this  GUI  panel  we  can  choose  various  parameters 
such  as  dot  movement  directions,  dot  size,  etc. 


Fig.  14  Preliminary  version  of  a  GUI  panel  to  control  the  characteristics  of  some  representational  strategies 
for  polarization  mapping. 


Summary 

In  this  project,  we  explored  and  developed  certain  representational  schemes  for  mapping 
polarization  information  into  visual  information.  In  particular,  in  this  effort  we  focused 
on  mapping  polarization  information  into  coherent  motion  and  into  static  and  dynamic 
textures  as  some  of  visual  cues  suitable  for  such  mapping.  Our  results  have  shown  that 


these  visual  cues  can  be  interesting  and  exciting  tools  in  representing  polarization 
information. 
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